主动学习(AL)是一个有希望的ML范式,有可能解析大型未标记数据并有助于降低标记数据可能令人难以置信的域中的注释成本。最近提出的基于神经网络的AL方法使用不同的启发式方法来实现这一目标。在这项研究中,我们证明,在相同的实验环境下,不同类型的AL算法(基于不确定性,基于多样性和委员会)产生了与随机采样基线相比的不一致增长。通过各种实验,控制了随机性来源,我们表明,AL算法实现的性能指标方差可能会导致与先前报道的结果不符的结果。我们还发现,在强烈的正则化下,AL方法在各种实验条件下显示出比随机采样基线的边缘或没有优势。最后,我们以一系列建议进行结论,以了解如何使用新的AL算法评估结果,以确保在实验条件下的变化下结果可再现和健壮。我们共享我们的代码以促进AL评估。我们认为,我们的发现和建议将有助于使用神经网络在AL中进行可重复的研究。我们通过https://github.com/prateekmunjal/torchal开源代码
translated by 谷歌翻译
Dense prediction tasks such as segmentation and detection of pathological entities hold crucial clinical value in the digital pathology workflow. However, obtaining dense annotations on large cohorts is usually tedious and expensive. Contrastive learning (CL) is thus often employed to leverage large volumes of unlabeled data to pre-train the backbone network. To boost CL for dense prediction, some studies have proposed variations of dense matching objectives in pre-training. However, our analysis shows that employing existing dense matching strategies on histopathology images enforces invariance among incorrect pairs of dense features and, thus, is imprecise. To address this, we propose a precise location-based matching mechanism that utilizes the overlapping information between geometric transformations to precisely match regions in two augmentations. Extensive experiments on two pretraining datasets (TCGA-BRCA, NCT-CRC-HE) and three downstream datasets (GlaS, CRAG, BCSS) highlight the superiority of our method in semantic and instance segmentation tasks. Our method outperforms previous dense matching methods by up to 7.2 % in average precision for detection and 5.6 % in average precision for instance segmentation tasks. Additionally, by using our matching mechanism in the three popular contrastive learning frameworks, MoCo-v2, VICRegL and ConCL, the average precision in detection is improved by 0.7 % to 5.2 % and the average precision in segmentation is improved by 0.7 % to 4.0 %, demonstrating its generalizability.
translated by 谷歌翻译
Indian e-commerce industry has evolved over the last decade and is expected to grow over the next few years. The focus has now shifted to turnaround time (TAT) due to the emergence of many third-party logistics providers and higher customer expectations. The key consideration for delivery providers is to balance their overall operating costs while meeting the promised TAT to their customers. E-commerce delivery partners operate through a network of facilities whose strategic locations help to run the operations efficiently. In this work, we identify the locations of hubs throughout the country and their corresponding mapping with the distribution centers. The objective is to minimize the total network costs with TAT adherence. We use Genetic Algorithm and leverage business constraints to reduce the solution search space and hence the solution time. The results indicate an improvement of 9.73% in TAT compliance compared with the current scenario.
translated by 谷歌翻译
In molecular research, simulation \& design of molecules are key areas with significant implications for drug development, material science, and other fields. Current classical computational power falls inadequate to simulate any more than small molecules, let alone protein chains on hundreds of peptide. Therefore these experiment are done physically in wet-lab, but it takes a lot of time \& not possible to examine every molecule due to the size of the search area, tens of billions of dollars are spent every year in these research experiments. Molecule simulation \& design has lately advanced significantly by machine learning models, A fresh perspective on the issue of chemical synthesis is provided by deep generative models for graph-structured data. By optimising differentiable models that produce molecular graphs directly, it is feasible to avoid costly search techniques in the discrete and huge space of chemical structures. But these models also suffer from computational limitations when dimensions become huge and consume huge amount of resources. Quantum Generative machine learning in recent years have shown some empirical results promising significant advantages over classical counterparts.
translated by 谷歌翻译
A major direction in differentially private machine learning is differentially private fine-tuning: pretraining a model on a source of "public data" and transferring the extracted features to downstream tasks. This is an important setting because many industry deployments fine-tune publicly available feature extractors on proprietary data for downstream tasks. In this paper, we use features extracted from state-of-the-art open source models to solve benchmark tasks in computer vision and natural language processing using differentially private fine-tuning. Our key insight is that by accelerating training, we can quickly drive the model parameters to regions in parameter space where the impact of noise is minimized. In doing so, we recover the same performance as non-private fine-tuning for realistic values of epsilon in [0.01, 1.0] on benchmark image classification datasets including CIFAR100.
translated by 谷歌翻译
Empirical studies suggest that machine learning models trained with empirical risk minimization (ERM) often rely on attributes that may be spuriously correlated with the class labels. Such models typically lead to poor performance during inference for data lacking such correlations. In this work, we explicitly consider a situation where potential spurious correlations are present in the majority of training data. In contrast with existing approaches, which use the ERM model outputs to detect the samples without spurious correlations, and either heuristically upweighting or upsampling those samples; we propose the logit correction (LC) loss, a simple yet effective improvement on the softmax cross-entropy loss, to correct the sample logit. We demonstrate that minimizing the LC loss is equivalent to maximizing the group-balanced accuracy, so the proposed LC could mitigate the negative impacts of spurious correlations. Our extensive experimental results further reveal that the proposed LC loss outperforms the SoTA solutions on multiple popular benchmarks by a large margin, an average 5.5% absolute improvement, without access to spurious attribute labels. LC is also competitive with oracle methods that make use of the attribute labels. Code is available at https://github.com/shengliu66/LC.
translated by 谷歌翻译
很少有细粒度的分类和人搜索作为独特的任务和文学作品,已经分别对待了它们。但是,仔细观察揭示了重要的相似之处:这两个任务的目标类别只能由特定的对象细节歧视;相关模型应概括为新类别,而在培训期间看不到。我们提出了一个适用于这两个任务的新型统一查询引导网络(QGN)。QGN由一个查询引导的暹罗引文和兴奋子网组成,该子网还重新进行了所有网络层的查询和画廊功能,一个查询实习的区域建议特定于特定于特定的本地化以及查询指导的相似性子网络子网本网络用于公制学习。QGN在最近的一些少数细颗粒数据集上有所改善,在幼崽上的其他技术优于大幅度。QGN还对人搜索Cuhk-Sysu和PRW数据集进行了竞争性执行,我们在其中进行了深入的分析。
translated by 谷歌翻译
会员推理(MI)攻击突出了当前神经网络随机培训方法中的隐私弱点。然而,它为什么出现。它们仅是不完美概括的自然结果吗?在培训期间,我们应该解决哪些根本原因以减轻这些攻击?为了回答此类问题,我们提出了第一种解释MI攻击及其基于原则性因果推理的概括的方法。我们提供因果图,以定量地解释以$ 6 $攻击变体获得的观察到的MI攻击性能。我们驳斥了几种先前的非量化假设,这些假设过于简化或过度估计潜在原因的影响,从而未能捕获几个因素之间的复杂相互作用。我们的因果模型还通过共同的因果因素显示了概括和MI攻击之间的新联系。我们的因果模型具有很高的预测能力($ 0.90 $),即它们的分析预测与经常看不见的实验中的观察结果相匹配,这使得通过它们的分析成为务实的替代方案。
translated by 谷歌翻译
提出测试释放(PTR)是一个差异隐私框架,可符合局部功能的敏感性,而不是其全球敏感性。该框架通常用于以差异性私有方式释放强大的统计数据,例如中位数或修剪平均值。尽管PTR是十年前引入的常见框架,但在诸如Robust SGD之类的应用程序中使用它,我们需要许多自适应鲁棒的查询是具有挑战性的。这主要是由于缺乏Renyi差异隐私(RDP)分析,这是一种瞬间的私人深度学习方法的基础。在这项工作中,我们概括了标准PTR,并在目标函数界定全局灵敏度时得出了第一个RDP。我们证明,与直接分析的$(\ eps,\ delta)$ -DP相比,我们的RDP绑定的PTR可以得出更严格的DP保证。我们还得出了亚采样下PTR的算法特异性隐私扩增。我们表明,我们的界限比一般的上限和接近下限的界限要紧密得多。我们的RDP界限可以为PTR的许多自适应运行的组成而更严格的隐私损失计算。作为我们的分析的应用,我们表明PTR和我们的理论结果可用于设计私人变体,用于拜占庭强大的训练算法,这些变体使用可靠的统计数据用于梯度聚集。我们对不同数据集和体系结构的标签,功能和梯度损坏的设置进行实验。我们表明,与基线相比,基于PTR的私人和强大的培训算法可显着改善该实用性。
translated by 谷歌翻译
在本文中,我们询问视觉变形金刚(VIT)是否可以作为改善机器学习模型对抗逃避攻击的对抗性鲁棒性的基础结构。尽管较早的作品集中在改善卷积神经网络上,但我们表明VIT也非常适合对抗训练以实现竞争性能。我们使用自定义的对抗训练配方实现了这一目标,该配方是在Imagenet数据集的一部分上使用严格的消融研究发现的。与卷积相比,VIT的规范培训配方建议强大的数据增强,部分是为了补偿注意力模块的视力归纳偏置。我们表明,该食谱在用于对抗训练时可实现次优性能。相比之下,我们发现省略所有重型数据增强,并添加一些额外的零件($ \ varepsilon $ -Warmup和更大的重量衰减),从而大大提高了健壮的Vits的性能。我们表明,我们的配方在完整的Imagenet-1k上概括了不同类别的VIT体系结构和大规模模型。此外,调查了模型鲁棒性的原因,我们表明,在使用我们的食谱时,在训练过程中产生强烈的攻击更加容易,这会在测试时提高鲁棒性。最后,我们通过提出一种量化对抗性扰动的语义性质并强调其与模型的鲁棒性的相关性来进一步研究对抗训练的结果。总体而言,我们建议社区应避免将VIT的规范培训食谱转换为在对抗培训的背景下进行强大的培训和重新思考常见的培训选择。
translated by 谷歌翻译